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Control Apparatus 

The present invention reLates to control apparatus and in particular to control apparatus for 
the remote control of data or equipment. 

There are many situations in which wireless remote control of equipment or data is 
5 particularly desirable, In many of these situations it would be particularly advantageous for 
ccmtrol to be dependent on the two or three dimensional position of a free moving control 
device relative to a control panel or the like. 

Three dimensional remote control may be particularly advantageous, for example, for 
improving the quality oflife of severely disabled people. Such controlroay be used by people 
10 withsevere mobility impairment, for operating special needs vehicles, household equipment 
computers or the like. Three dimensional remote control would, allow the minimum effort 
with no pressure being required to operate switches or the like. Conveniently, the control 
device could be configured to the needs of the disabled person allowing Mm to wear the 
device in a convenient position for mobility. 

1 5 Three dimensional remote conlrol also has applications in three dimensional design. It would 
be particularly beneficial; for example, for a designer to be able to create a 'virtual' 3D design 
by tracing a three dimensional image in mid-air above a. control panel in the form of a sensor 
plate. A computer design system integrating a wireless, freely held control device in the form 
of a stylus or the like would allow a more seamless, intuitive, and artistic effort. 

20 It is an object of the invention to provide an improved control apparatus for the remote 
control of data or equipment. 

According to one aspect of the invention there is provided control apparatus for the remote 
control of data or equipment, comprising; a screen; inputmeans; sensing means for sensing 
data related to a position of the input means relative to the screen; control means for 
£5 determining the position of the input means relative to said screen from the data; wherein, the 
inputmeans includes a source of electromagnetic radiation for directing onto the screen; the 
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sensing means includes detection means for detecting electromagnetic radiation directed 
onto, and scattered by the screen; the detection means being configured to produce a detector 
output dependent on the scattered electro-magnetic radiation detected; and the control means 
being configured to analyse the detector output to determine the position of the source 
5 relative to the screen, 

Preferably the source is configured to produce a conical beam of infrared radiation for 
directing onto the screen to form an elliptical area of incident radiation* 

The source maybe configured to produce a laser beam for directing onto the screen. 

Preferably the control means is configured to analyse the detector output, to measure the 
1 0 length of either a major axis or a minor axis of the incident elliptical area, and to calculate the 
distance between the source and the screen from said measured length. 

Preferably ,the control means is configured to analyse the detector output to measure angle 
related parameters of electro-magnetic radiation directed by the source onto the screen, and to 
determine the angle of the source relative to screen from the measured parameters. 



1 5 Preferably the area related parameters include the I ength of either a major axis or a minor axis j 
of the incident elliptical area. j 

Preferably the source is configured to split the laser beam into a plurality of components for 
directing onto the screen to form a plurality of corresponding incident points. 

Preferably the control means is configured to analyse the detector output to measure the 
20 distances between the incident points. 

The control means may be configured to determine the distance of the source from the screen 
from at least one of the measured distances. 



The control means may be configured to determine the angle of the source relative to the 
screen ftom the measured distances. 
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Preferably the detection means comprises a plurality of detectors arranged along at least one 
edge of the screen; each detector being configured to produce an analogue output dependent 
on the quantity of scattered light detected by it 

Preferably the detection means is configured to produce detector output in the form of aserial 
5 signal, and the detection means further comprises a converter configured to convert the 
analogue outputs into the serial signal. 

According to another aspect of the invention there is provided a method of detennining a 
position of an input means relative to a screen comprising: directing electromagnetic 
radiation onto the screen from the input device; detecting electromagnetic radiation directed 
10 onto, and scattered by the screen; producing a detector output dependent on scattered electro- 
magnetic radiation detected; and analysing the detector output to determine the position of 
the input device relative to the screen. 

The present invention will now be described, by way of example only, with reference to the 
accompanying drawings in which: 

15 Figure 1 shows illustratively a device incorporating the invention; 

Figure 2 shows illustratively a screen in use with apointer according to one embodiment of 
the invention; 

Figure 3 shows illustratively a plan view of the screen of figure 2 

Figure 4 shows illustratively a pointer according to another embodiment of the invention in 
20 operation; 

Figure 5 shows illustratively the screen in use with the pointer of figure 4; and 



Figure 6 shows illustratively a plan view of the screen of figure 5, 

Referring to Figure 1 , an electronic device is shown generally at 10. The device 1 0 may be, 
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for example, a hand-held or "palm-top" computer, a personal digital assistant (PDA), a PC 
screen, or a mobile communication device such as a mobile telephone. Alternatively the 
device may be an electronic control panel for controlling electronic equipment such as 
surgical apparatus, an electric vehicle or the like. 

5 The device 10 has a screen 12, for displaying data, or control options for controlling 
electronic equipment. In operation, the data displayed or the electronic equipment is 
controlled by means of an input device, bi the preferred embodiment the input device takes 
the foim of a pointer 16, which may be of any suitable form, for example a pen-shaped 
"stylus", a glove, a visor, a gun, or any suitable remote unit The pointer 1 6 allows the user to 
10 operate the device 10 by selecting various options displayed on the, screen 12. 

The screen 12 is made of a relatively, transparent material with small but significant 
scattering properties, for example, commercial grade Perspex or the like. Hence, in operation, 
when the pointer 16 emits a beam of light onto a region of the screen 12 most of the light 
incident on the screen 12 is transmitted through the screen 12 and some of the light is 
15 scattered towards the edges of the screen 12. The quantity of transmitted light passing 
through the screen 12 decreases exponentially with the thickness of the screen 12. The 
quantity of scattered light arriving at the edges of the screen decreases exponentially with 
distance from the region of incident light. 

As shown in figures 2 and 5, the screen 12 includes a plurality of horizontal detectors 20, and 
20 vertical detectors 22, arranged along the edges of the screen 12 in the x and y directions 
respectively. The detectors 20, 22 are configured to detect the scattered light and typically 
have a relatively narrow field of view, for example ~S°. They may be of any form suitable for 
detecting electromagnetic radiation of the type emitted by the pointer, for example, 
conventional photo detectors. 

25 The pointer 16 is provided with a source of electromagnetic radiation, for example an 
infrared emitter, a laser, an LED or other such light emitting device. 

In figure 3 a pointer according one embodiment is designated 16'. The pointer 16' emits a 
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conical beam of light, as best illustrated in figure 3, £x>m a circular, spherical* or other shaped 
tip (not shown). The light emitted is modulated in order to avoid interference from ambient 
light. 

In operation, when the pointer 16* emits a beam of light onto a region of the screen 12, each 
5 detector 20, 22 detects a portion of ths light scattered from the incident beam and produces a 
corresponding analogue output 24, 24' with a magnitude dependent on the quantity of 
scattered light detected. The analogue outputs 24, 24' are received in parallel by a converter 
26, 26% which pit>duces a corresponding serial output 28, 28\ The serial output 28, 28 s i$ 
inputted to the device 10 which analyses the signal to determine pointer parameters, for 
1 0 example the relative X, Y and Z position and/or angle of inclination of the pointer 1 6. 

The converter 24, 24' is of any suitable type and may be an analogue to digital converter. The 
serial output may be irx the form of a digital serial signal or an analogue serial signal 

The horizontal detector 20, and vertical detector 22, with the highest magnitude analogue 
output correspond respectively to the x and y positions of the incident beam. Hence, hi 
1 5 operation, the device 10 measures the magnitude of the analogue outputs, determines the x 
and y position of the incident beam from the measured values, and responds appropriately. 

hi operation, the accuracy of the position determined is improved by interpolation between 
the values of the outputs measured for multiple detectors. In each direction the value 
measured for the detector with the highest magnitude output is interpolated with the values 
20 measured for adjacent detectors using a polynomial function. The polynomial function is 
quadratic having the form: 

V(x) = ax 2 4- bx + c for the x direction 

and 

V(y) = a'y 2 + b'y + c* for the y direction, 

25 where the functions V(x) and V(y) are the values of the outputs of the detectors at position K 
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and position y respectively. The constants a s a', b, b* and c ? c 9 are determined by finding the 
best-fit to the measured values, The interpolated test value is approximated by: 

x = -b/2a for the x position 

and 

5 y ~ -b72a ? for the y position 

Hence, a more accurate value of x and y position are found. 

The region of the screen 1 2 on which the light is incident will be substantially elliptical with 
an area dependent upon the distance of the pointer 16 ? from the screen. The eccentricity of 
the ellipse -will depend on the angle at which the beam strikes the screen and hence on the 
10 angle at which the pointer is held An eccentricity of 1, for example, is indicative of a circle 
of incident light and the pointer 1 & held perpendicular to the surface of the screen. 

The relative magnitudes of the analogue outputs 24, 24 ? are used to determine the lengths of 
the major and misior ellipse axis and hence the eccentricity the ellipse, The angle of the 
pointer 16' is then calculated using conventional mathematics. Similarly the length of either 
15 the major or minor ellipse axis is used to calculate the distance of the pointer from the screen 
in the Z direction. Where the ellipse is a circle, the length of both ellipse axis will be equal, 
corresponding to the diameter d of the circle as seen in figure 3. 

The ratio of the constants a to a 3 derived from the interpolation is also a measure of the angle 
of the pointer 16* Tdative to the screen 12, Hence, in operation the constants derived from 
20 interpolation may also be used to improve the accuracy of the angle determined. 

The intensity of the light incident on the screen is also dependent on the distance of the 
pointer W from the screen 12. Correspondingly, the quantity of scattered light detected by 
the detectors 20, 22, and hence the magnitudes of the analogue signals 24, 24' vary with the 
intensity of the incident light. Hence, the absolute magnitudes of the analogue signals 24, 24' 
25 may also be used ? either to independently determine the distance of the pointer 1 6' from the 
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screen 12, or to refine the result of the calculation baaed on the ellipse measurements. 

The distance of the pointer 16' from the screen 12 is also generally inversely proportional to 
the constants a and a' derived from the interpolation. Hence, in operation the constants 
derived from interpolation may also he used to improve the accuracy of the z position 
5 determined. 

The angle of the pointer 1$% in conjunction with the distance of the pointer 16' from the 
screen 12 are then used to determine the distance of the pomter-16* from the screen 12. Hence 
the position of the pointer 16% in the Z-dimension, is determined by the device 10. 

The location and angle of the pointer 1 6\ may also be used to determine when the user makes 
10 a selection without physical contact between the pointer 1 6' and the screen. A simple dipping 
motion, for example, could be used to represent the selection. Alternatively or additionally 
the area and / or intensity of the light may also be used to represent a contactless selection. 
Such a selection may be indicated, for example, by the area of incident light falling below a 
certain minimum threshold and / or the intensity rising above a certain maximum threshold. 

15 Figure 4 shows apoitfter 16" according to another embodiment of the mvention> The pointer 
16" includes a laser source 40, three beam splitters 42, 44, 46 and a window 48. The laser 
source may be of any suitable type, for example, a laser diode or the like. 

The laser source 40 is located within the pointer 16" and is configured to produce a laser 
beam 50 in an axial direction of the laser source 40. The beam splitters 42, 44, 46 are in the 

20 form of planar, semi-reflecting mirrors arranged within the pointer 1 6' 'arranged at intervals 
along the operational path of the laser beam 50. Each beam splitter 42, 44, 46 is positioned at 
an angle a/2 relative to the axial direction of the laser source 40. Hence, in operation each 
beam splitter 42, 44, 46 splits the laser beam 50 into a first component, which passes through 
the splitter with no change in direction, and a second component, which is deflected through 

25 an angle a. 

The first, second and third beam splitters 42, 44, 46 are located progressively further from the 
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laser source 40, The second and third beam splitters 44, 46 are respectively rotated through 
120°, and -120*, relative to the first beam splitter 42 about the axial direction of the laser 
source 40. Hence, in operation the laser source produces the laser beam 50^ which passes 
through each beam splitter 42, 44, 46, in turn. Thus, the beam is split into four components 
5 comprising three deflected components 52, 54, 56 and a transmitted component S3. As seen 
in figure 5 and 6 ? in operation, the deflected components 52, 54, 56 incident on the screen 12, 
form the vertices of a triangle. The incident transmitted component 58 lies substantially at the 
centre of the triangle, 

Each beam splitter 42 ? 44, 46, may be made of any suitable material, for example, microscope 
1 0 cover glass, The Splitters 42, 44, 46, are each provided with a thin-film optical interference 
coating for controlling the relative transmittance and reflectance of each splitter 42, 44, 46 
and hence the relative intensity of the deflected and transmitted components 52, 54, 56, 58. 
Alternatively of additionally the pointer 16" may be provided with an absorbing filter 60 
located in the operational path of the transmitted component 58 for attenuating the intensity 
15 of the transmitted component 5 S relative to the deflected components 52, 54, 56. 

Thus, in operation, the intensity of the transmitted component 58 is different to the intensity 
of the deflected components 52> 54, 56. Hence, when the components of the laser beam 50 
are incident on the screen 12, the transmitted component 5S can be distinguished from the 
deflected components 52 a 54, 56. 

20 By way of example, when the beam splitters 42, 44, 46 are made of microscope cover glass 
and a is set to approximately 13 Q , the associated ratio of transmittance to reflectance is 
approximately 1:1. Hence, the intensity of the transmitted component 58 emerging from the 
third beam splitter 46 is approximately: 25% of the intensity of the first deflected component 
52, deflected by the fiirst beam splitter 42; 50% of the intensity of the second deflected 

25 component 54„ deflected by the second beam splitter 44; and equal to the intensity of the third 
deflected component 56 3 deflected by the third beam splitter 46, The absorbing filter 60 
reduces the intensity of the transmitted component 58 further so that it is less than that of the 
other components. 
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The window 48 is located at the end of the pointer 16" in the operational path of the 
transmitted component 58 of the laser beam 50. The window 48 is of suitable size and 
configuration to allow passage of all three deflected components 52, 54, 56 and the 
transmitted component 58 during operation. 

5 The use of the laser source 40 is particularly advantageous for operation of the device 10 
from a distance, since laser beams have extremely narrow spreads and maintain a relatively 
high intensity over long distances. It will be appreciated that the precise location and 
arrangement of the laser source 40 and beam splitters may vary dependant on requirements. 
For example, where remote operation is required from a distance, the angle a may be 

10 particularly acute. 

Operation to determine the three dimensional position and orientation of the pointer 16' ' is 
similar to that described for the pointer 1 6' having a conical beam. 

In operation, when the pointer 16' ' emits the components of the laser beam 50 onto a region 
of the screen 12, a number of the detectors 20, 22 detect light scattered from the i&eident 

15 components atid produce a corresponding analogue output 24, 24' with a magnitude 
dependent on the quantity of scattered light detected. The analogue outputs 24, 24 s are 
received in parallel by a converter 26, 26% which produces a corresponding serial output 28, 
28\ The serial output 28, 28' is inputted to the device 10 which analyses the signal to 
deteuftine pointer parameters, for example the relative X ? Y and Z position and/or angle of 

20 inclination of the pointer 16". 

The X position of each component is determined from the relative magnitudes of the signals 
24 from the horizontal detectors 20. Similarly the Y position, of each component is 
determined from the relative magnitudes of the signals 24' from the vertical detectors 22. As 
discussed the intensity of the transmitted component 58 is different to that of the other 
25 components of the laser beam 50. Hence, the X and Y positions corresponding to the incident 
transmitted component 58 is distinguished from the positions of the other components and 
the device 10 can respond appropriately, for example, by highlighting an icon at which the 
pointer is directed. 
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Referring to figure 6 the distance between the transmitted and deflected components 52, 54, 
56, 58 incident on the screen 12 is determined from their relative positions and hence the 
distance in the Z direction can also be detennined by conventional trigonometry, 

As with LED light the intensity of laser light incident on the screen is also dependent on, the 
5 distance of the pointer 16** from the screen 12. Coocespondingly, the quantity of scattered 
light detected by the detectors 20, 22, and hence the magnitudes of the analogue signals 24, 
24* vary with the intensity of the incident laser light. Hence, the absolute magnitudes of the 
analogue signals 24, 24' may also be used, either to independently detemiine the distance of 
the pointer 16" from the screen 12, or to refine the result of the calculation based on the 
10 trigonometrical calculations. 

The relative distances between the transmitted and deflected components 52, 54, 56, 58, 
incident on the screen 12 is indicative of the equality of the triangle formed by the incident 
deflected component* 52, 54, 56, For example, when the pointer 16 is held with the 
transmitted component 53 perpendicular to the screen 12, the triangle formed by the deflected 
1 5 components 52, 54, 5$, is substantially equilateral When the transmitted component 58 is not 
normal to the screen 12, the triangle formed is not equilateral and the distances between the 
components reflect this. Hence, the angle at which the pointer 1 6 is being held relative to the 
screen 12 is determined using conventional trigonometry. 

The angle of the pointer 16' \ in conjunction with the distance of the pointer 16' ' from the 
20 screen 12 are used to determine the distance of the pointer 16" from the screen 12. Hence the 
position of the pointer 1 6' \ in the ^dimension, is determined by the device 10. 

Repetitive calculation of the pointer position, several times a second as the pointer W \ is 
moved allows a pointer trajectory to be recorded. The pointer trajectory may then be used to 
assist in anticipating the intentions of the user or for advance control functions. 

25 , The location and angle of the pointer 16'\ may also be used to determine when the user 
makes a selection without physical contact between the pointer 16" and the screen. A simple 
dipping motion, for example, could be used to represent the selection, 
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It will be appreciated that although the invention has been described with reference to an 
embodiment in which the three dimensional position of a pointer may be determined, the 
invention is equally applicable for determining the position of the pointer in two dimensions 
only or even a single dimension. 

5 It will further be appreciated that although the invention has been described with reference to 
particular applications there are numerous other ways in which the invention maybe applied. 
A number of applications will now be described for the purposes of illustration. 

The invention has particular application in the medical profession. For example, for 
computer-aided surgical procedures such as kparoscopy, microsurgery, and endoscopy, hi 
1 0 this case the light source could be built into tip of a pointer in the form of a scalpel or other 
tool for a more natural feel for a doctor or surgeon. 

In telemedicine the invention maybe used for the control of cameras and medical equipment 
in remote medical examinations or procedures. 

The invention may also be incorporated into equipment for training purposes. For example, it 
1 5 would be particularly beneficial in medical student training, for allowing simulated medical 
procedures to be carried out on computerized dummies such as CPR training mannequins or 
using other computer controlled simulators* A pointer in the form of a light-tipped surgical 
tool could be used to enhance the realism of the simulation making it more effective. 
Potentially such use could also reduce the requirements for cadavers in medical training. 

20 The invention would also be of advantage in applications where there is a requirement for the 
human operation of equipment based in hazardous environments such as the vacuum of space 
or military settings. Examples of such uses include the intuitive control of unmanned 
equipment and the external repair and adjustment of space based vehicles and equipment. A 
single hardware system could therefore be programmed with separate programming 

25 preferences for each piece of equipment, minimizing weight and equipment limitations on 
board. 
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The invention also has many applications in small scale research, engineering and 
manufacturing including biotechnology, biology, semiconductors and nanotechtiology. In 

situations where the operator is working with items too small to be seen by the naked eye, a 

system can be devised to integrate the pointer with a computerised microscope, for example, 
5 an electron-microscope or the like, The microscopic real-time image c<m then be used in 

conjunction with the pointer to control the microscope and to guide manipulate a 
'workpiece 7 . In this application it would be particularly beneficial if Hie light source were a 

laser since this can be focussed extremely precisely. 

Beneficial functionality may include, for example, simultaneous zoom control in conjunction 
1 0 with active functions in a single wireless pointer in the form of a stylus or other tool The 
pointer being used directly "on" a microscope image for designing microchips, investigating 
viruses or genes, 

The invention may also "be incorporated into equipment for fertility treatments, such as ICSI 
(intracytoplssmte sperm injection). In ICSI a microscopically tipped pipette is used to select a 
15 single speim md to inject it into an egg. Having the pipette controlled by an operator holding 
a pointer in the form of a computer-driven, light-tipped "pipette'^and working directly on the 
microscope-generated image would have obvious advantages. The same principle applies to 
ex vivo treatment (IVF), colonoscopy, and other invasive biopsy investigations, where trauma 
to a patient needs to be minimised. 

20 In all these situations the pointer is a hand held tool which does not come into direct contact 
with the work in progress. This allows appropriate preferences and tolerances to be set to 
reduce the probability and magnitude of errors caused by tinder-compensation, over- 
compensation or undesirable movements, 

A further application of the invention is for the enhanced control of televisions and other 
25 entertainment equipment. A light-sensitive screen in combination w r itji an appropriately 
configured pointer could be used in a way similar to current remote-control systems, but with 
additional enhancements. For example, adjustability of a tc frame within a frame" could be 
provided with ''click and drag" being used to adjust the size of the second frame. Similarly, 
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provision could be made for direct on-screen programming of a VCR, other recording device 
or integrated, media. 

A particular advantage in these applications is theprc-vision of button-free menu operation, 
allowing for a mors user-friendly and truly interactive procedure. 
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Claims 

1 Control apparatus for the remote control of data or equipment, comprising: 
a screen; 

input means; 

5 sensing means for sensing data related to a position of the input means relative to the screen; 

control means for determining the position of the input means relative to said screen from the 
data; 

wherein; 

the input means includes a source of electromagnetic radiation for directing onto the screen; 

1 0 the sensing means includes detection means for detecting electromagnetic radiation directed 
onto 3 and scattered by the screen; 

the detection means is configured to produce a detector output dependent on scattered 
electro-magnetic radiation detected; f 

! 

and the control means is configured to analyse the detector output to determine the position j 
15 of the source relative to the screen. j 

! 

i 

i 

2 An electronic device as claimed in claim 1 wherein the source is configured to j 
produce a conical beam of infrared radiation for directing onto the screen to foam an elliptical ! 
area of incident radiation. 

I 

3 An electronic device as claimed in claim 1 wherein the source is configured to j 

I 

20 produce a laser beam for directing onto the screen. 

i 

4 An electronic device as claimed in claim 2 wherein ffie control means is configured to 
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analyse the detector output, to measure the length of either a maj or axis or a minor axis of the 
incident elliptical area, and to calculate the distance between the source md the screen from 
said measured length. 

5 An electronic device as claimed in claim 2 or 4 wherein the contra 1 means is 
S configured to analyse the detector output to measure angle related parameters of electro- 
magnetic radiation directed by the source onto the screen, and to determine the angle of the 
source relative to screen from the measured parameters. 

6 An electronic device as claimed in claim 5 wherein the area related parameters 
include the length of either a. major axis or a minor axis of the incident elliptical area. 

10 7 An electronic device as claimed in claim 3 wherein the source is configured to split 
the laser "beam into a plurality of components for directing onto the screen to form aplurality 
of corresponding incident points, 

S An electronic device as claimed in claim 7 wherein the control means is configured to 
analyse the detector output to measure the distances between the incident points. 

15 9 An electronic device as claimed in claim 7 wherein the control means is configured to 
determine the distance of the source from the screen, from at least one of the measured 
distances. 

10 An electromc device as claimed in claim 8 or 9 wherein the control means is 
configured to determine the angle of the source relative to the screen from the measured 

20 distances. 

1 1 An electronic device as claimed in any preceding claim wherein the detection means 
comprises a plurality of detectors arranged along at least one edge of the screen; each detector 
being configured to produce an analogue output dependent on the quantity of scattered light 
detected by it 

25 12 An electronic device as claimed in claim 10 wherein the detection means is 
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configured to produce detector output in the form of a serial signal and the detection means 
further comprises a converter configured to convert the analogue outputs into the serial 
signal. 

13 A method of determining a position of an input means relative to a screen 
5 comprising: 

directing electromagnetic x^diation onto the screen from the input device; 

detecting electromagnetic radiation directed onto^ and scattered by the screen; 

producing a detector output dependent on scattered electro-magnetic radiation detected; and 

analysing the detector output to determine the position of the input device relative to the 
10 screen. 
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Fig 2 



I.APR. 2004 1 2:36 LEWIS & TAYLOR MO. 0201 P. 23 




3/< 




•JO- 



nnnnnnnnn 



[009961.5 PT-Apr-Q4l2°:58 1 



1. APR. 2004 12:36 LEWIS & TAYLOR NO. 0201 P, 24 





Fig 4 
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